Representations and representation learning for image aesthetics prediction and image enhancement

With the continual improvement in cell phone cameras and improvements in the connectivity of mobile devices, we have seen an exponential increase in the images that are captured, stored and shared on social media. For example, as of July 1st 2017 Instagram had over 715 million registered users which had posted just shy of 35 billion images. This represented approximately seven and nine-fold increase in the number of users and photos present on Instagram since 2012. Whether the images are stored on personal computers or reside on social networks (e.g. Instagram, Flickr), the sheer number of images calls for methods to determine various image properties, such as object presence or appeal, for the purpose of automatic image management and curation. One of the central problems in consumer photography centers around determining the aesthetic appeal of an image and motivates us to explore questions related to understanding aesthetic preferences, image enhancement and the possibility of using such models on devices with constrained resources. In this dissertation, we present our work on exploring representations and representation learning approaches for aesthetic inference, composition ranking and its application to image enhancement. Firstly, we discuss early representations that mainly consisted of expert features, and their possibility to enhance Convolutional Neural Networks (CNN). Secondly, we discuss the ability of resource-constrained CNNs, and the different architecture choices (inputs size and layer depth) in solving various aesthetic inference tasks: binary classification, regression, and image cropping. We show that if trained for solving fine-grained aesthetics inference, such models can rival the cropping performance of other aesthetics-based croppers, however they fall short in comparison to models trained for composition ranking. Lastly, we discuss our work on exploring and identifying the design choices in training composition ranking functions, with the goal of using them for image composition enhancement

Dry Etching for High Aspect Ratio Silicon Fins

The aim of this project is to develop a dry etching process that could be utilized to realize high aspect ratio fins. The traditional planar silicon transistor has seen amazing development ever since it became the workhorse of the semi conductor industry and the dominant way of realizing logic and analog circuits. However as its gate channels length is scaled down, the transistor encounters issues termed short channel effects and which has sprouted an extensive research into future replacements for the planar MOSFET. One of such devices is the Fin Field Effect Transistor or FinFET. Most of the FinFET fabrication processes rely heavily on a highly anisotropic etching process to realize very thin fins. This report details the investigation into the possible chemistries used for anisotropic etching, further exploration and improvement of the process on wafers pieces. In the end, the report describes the transfer of the Silicon etch from the wafer pieces to a full six inch wafer

DeepPatent2: A Large-Scale Benchmarking Corpus for Technical Drawing Understanding

Recent advances in computer vision (CV) and natural language processing have been driven by exploiting big data on practical applications. However, these research fields are still limited by the sheer volume, versatility, and diversity of the available datasets. CV tasks, such as image captioning, which has primarily been carried out on natural images, still struggle to produce accurate and meaningful captions on sketched images often included in scientific and technical documents. The advancement of other tasks such as 3D reconstruction from 2D images requires larger datasets with multiple viewpoints. We introduce DeepPatent2, a large-scale dataset, providing more than 2.7 million technical drawings with 132,890 object names and 22,394 viewpoints extracted from 14 years of US design patent documents. We demonstrate the usefulness of DeepPatent2 with conceptual captioning. We further provide the potential usefulness of our dataset to facilitate other research areas such as 3D image reconstruction and image retrieval

Monitoring a photovoltaic system during the partial solar eclipse of August 2017

The power output of a 4.85 kW residential photovoltaic (PV) system located in Rochester, NY is monitored during the partial solar eclipse of August 21, 2017. The data is compared with the data on a day before and on the same day, a year ago. The area of exposed solar disk is measured using astrophotography every 16 s of the eclipse. Global solar irradiance is estimated using the eclipse shading, time of the day, location coordinates, atmospheric conditions and panel orientation. A sharp decline, as expected in the energy produced is observed at the time of the peak of the eclipse. The observed data of the PV energy produced is related with the model calculations taking into account solar eclipse coverage and cloudiness conditions. The paper provides a cohesive approach of irradiance calculations and obtaining anticipated PV performance

Monitoring a photovoltaic system during the partial solar eclipse of August 2017

The power output of a 4.85 kW residential photovoltaic (PV) system located in Rochester, NY is monitored during the partial solar eclipse of August 21, 2017. The data is compared with the data on a day before and on the same day, a year ago. The area of exposed solar disk is measured using astrophotography every 16 s of the eclipse. Global solar irradiance is estimated using the eclipse shading, time of the day, location coordinates, atmospheric conditions and panel orientation. A sharp decline, as expected in the energy produced is observed at the time of the peak of the eclipse. The observed data of the PV energy produced is related with the model calculations taking into account solar eclipse coverage and cloudiness conditions. The paper provides a cohesive approach of irradiance calculations and obtaining anticipated PV performance

DeepPatent: Large Scale Patent Drawing Recognition and Retrieval

We tackle the problem of analyzing and retrieving technical drawings. First, we introduce DeepPatent, a new large-scale dataset for recognition and retrieval of design patent drawings. The dataset provides more than 350,000 design patent drawings for the purpose of image retrieval. Unlike existing datasets, DeepPatent provides fine-grained image retrieval associations within the collection of drawings and does not rely on cross-domain associations for supervision. We develop a baseline deep learning models, named PatentNet, based on best practices for training retrieval models for static images. We demonstrate the superior performance of PatentNet when trained on our fine-grained associations of DeepPatent against other deep learning approaches and classic computer vision descriptors, such as histogram of oriented gradients (HOG), on DeepPatent. With the introduction of this new dataset, and benchmark algorithms, we demonstrate that the analysis and retrieval of line drawings remains an open challenge in computer vision; and that patent drawing retrieval provides a concrete testbench to spur research